Collaborative Reinforcement Learning for Multi-Service Internet of Vehicles

Internet of Vehicles (IoV) is a recently introduced paradigm aiming at extending the Internet of Things (IoT) toward the vehicular scenario in order to cope with its specific requirements. Nowadays, there are several types of vehicles, with different characteristics, requested services, and delivered data types. In order to efficiently manage such heterogeneity, Edge Computing facilities are often deployed in the urban environment, usually co-located with the Roadside Units (RSUs), for creating what is referenced as Vehicular Edge Computing (VEC). In this paper, we consider a joint network selection and computation offloading optimization problem in multi-service VEC environments, aiming at minimizing the overall latency and the consumed energy in an IoV scenario. Two novel collaborative Q-learning based approaches are proposed, where Vehicle-to-Infrastructure (V2I) and Vehicle-to-Vehicle (V2V) communication paradigms are exploited, respectively. In the first approach, we define a collaborative Q-learning method in which, through V2I communications, several vehicles participate in the training process of a centralized Q-agent. In the second approach, by exploiting the V2V communications, each vehicle is made aware of the surrounding environment and the potential offloading neighbors, leading to better decisions in terms of network selection and offloading. In addition to the tabular method, an advanced deep learning-based approach is also used for the action value estimation, allowing to handle more complex vehicular scenarios. Simulation results show that the proposed approaches improve the network performance in terms of latency and consumed energy with respect to some benchmark solutions

A Markov Decision Process Solution for Energy-Saving Network Selection and Computation Offloading in Vehicular Networks

Vehicular Edge Computing (VEC) enables the integration of edge computing facilities in vehicular networks (VNs), allowing data-intensive and latency-critical applications and services to end-users. Though VEC brings several benefits in terms of reduced task computation time, energy consumption, backhaul link congestion, and data security risks, VEC servers are often resource-constrained. Therefore, the selection of proper edge nodes and the amount of data to be offloaded becomes important for having VEC process benefits. However, with the involvement of high mobility vehicles and dynamically changing vehicular environments, proper VEC node selection and data offloading can be challenging. In this work, we consider a joint network selection and computation offloading problem over a VEC environment for minimizing the overall latency and energy consumption during vehicular task processing, considering both user and infrastructure side energy-saving mechanisms. We have modeled the problem as a sequential decision-making problem and incorporated it in a Markov Decision Process (MDP). Numerous vehicular scenarios are considered based upon the users' positions, the states of the surrounding environment, and the available resources for creating a better environment model for the MDP analysis. We use a value iteration algorithm for finding an optimal policy of the MDPs over an uncertain vehicular environment. Simulation results show that the proposed approaches improve the network performance in terms of latency and consumed energy

Joint Air-Ground Distributed Federated Learning for Intelligent Transportation Systems

Supported by some of the major revolutionary technologies, such as Internet of Vehicles (IoVs), Edge Computing, and Machine Learning (ML), the traditional Vehicular Networks (VNs) are changing drastically and converging rapidly into one of the most complex, highly intelligent, and advanced networking systems, mostly known as Intelligent Transportation System (ITS). Recently, distributed ML techniques, such as Federated Learning (FL) have gained huge popularity mainly for their advantages in terms of intelligence sharing and privacy concerns. VNs are a natural contender for exploiting FL for solving challenging problems; however, their limited resources, dynamic nature, high speed, and reduced latency requirements often become the bottleneck. V2X communication technologies allow vehicular terminals (VTs) to share their valuable local environment parameters and become aware of their surroundings. Such information can be utilized to build a more sustainable and affordable FL platform for serving VTs. Gaining from recently introduced 3D architectures, integrating terrestrial and aerial edge computing layers, we present here a distributed FL platform able to distribute the FL process on a 3D fashion while reducing the overall communication cost for providing vehicular services. The framework is defined as a constrained optimization problem for reducing the overall FL process cost through a proper network selection between various nodes. We have modeled the FL network selection problem as a sequential decision-making process through a Markov Decision Process (MDP) with time-dependent state transition probabilities. A computation-efficient value iteration algorithm is adapted for solving the MDP. Comparison with various benchmark methods shows the overall improvement in terms of latency, energy, and FL performance

Energy Efficient Adaptive Network Coding Schemes for Satellite Communications

In this paper, we propose novel energy efficient adaptive network coding and modulation schemes for time variant channels. We evaluate such schemes under a realistic channel model for open area environments and Geostationary Earth Orbit (GEO) satellites. Compared to non-adaptive network coding and adaptive rate efficient network-coded schemes for time variant channels, we show that our proposed schemes, through physical layer awareness can be designed to transmit only if a target quality of service (QoS) is achieved. As a result, such schemes can provide remarkable energy savings.Comment: Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, 24 March 201

Adaptive Network Coding Schemes for Satellite Communications

In this paper, we propose two novel physical layer aware adaptive network coding and coded modulation schemes for time variant channels. The proposed schemes have been applied to different satellite communications scenarios with different Round Trip Times (RTT). Compared to adaptive network coding, and classical non-adaptive network coding schemes for time variant channels, as benchmarks, the proposed schemes demonstrate that adaptation of packet transmission based on the channel variation and corresponding erasures allows for significant gains in terms of throughput, delay and energy efficiency. We shed light on the trade-off between energy efficiency and delay-throughput gains, demonstrating that conservative adaptive approaches that favors less transmission under high erasures, might cause higher delay and less throughput gains in comparison to non-conservative approaches that favor more transmission to account for high erasures.Comment: IEEE Advanced Satellite Multimedia Systems Conference and the 14th Signal Processing for Space Communications Workshop (ASMS/SPSC), 201

Network Coding Channel Virtualization Schemes for Satellite Multicast Communications

In this paper, we propose two novel schemes to solve the problem of finding a quasi-optimal number of coded packets to multicast to a set of independent wireless receivers suffering different channel conditions. In particular, we propose two network channel virtualization schemes that allow for representing the set of intended receivers in a multicast group to be virtualized as one receiver. Such approach allows for a transmission scheme not only adapted to per-receiver channel variation over time, but to the network-virtualized channel representing all receivers in the multicast group. The first scheme capitalizes on a maximum erasure criterion introduced via the creation of a virtual worst per receiver per slot reference channel of the network. The second scheme capitalizes on a maximum completion time criterion by the use of the worst performing receiver channel as a virtual reference to the network. We apply such schemes to a GEO satellite scenario. We demonstrate the benefits of the proposed schemes comparing them to a per-receiver point-to-point adaptive strategy

Wireless Communication Protocols for Distributed Computing Environments

The distributed computing is an approach relying on the presence of multiple devices that can interact among them in order to perform a pervasive and parallel computing. This chapter deals with the communication protocol aiming to be used in a distributed computing scenario; in particular the considered computing infrastructure is composed by elements (nodes) able to consider specific application requests for the implementation of a service in a distributed manner according to the pervasive grid computing principle (Priol & Vanneschi, 2008; Vanneschi & Veraldi, 2007). In the classical grid computing paradigm, the processing nodes are high performance computers or multicore workstations, usually organized in clusters and interconnected through broadband wired communication networks with small delay (e.g., fiber optic, DSL lines). The pervasive grid computing paradigm overcomes these limitations allowing the development of distributed applications that can perform parallel computations using heterogeneous devices interconnected by different types of communication technologies. In this way, we can resort to a computing environment composed by fixed ormobile devices (e.g., smartphones, PDAs, laptops) interconnected through broadband wireless or wired networks where the devices are able to take part to a grid computing process. Suitable techniques for the pervasive grid computing should be able to discover and organize heterogeneous resources, to allow scaling an application according to the computing power, and to guarantee specific QoS profiles (Darby III & Tzeng, 2010; Roy & Das, 2009). In particular, aim of this chapter is to present the most important challenges for the communication point of view when forming a distributed network for performing parallel and distributed computing. The focus will be mainly on the resource discovery and computation scheduling on wireless not infrastructured networks by considering their capabilities in terms of reliability and adaptation when facing with heterogeneous computing requests

Enabling Intelligent Vehicular Networks Through Distributed Learning in the Non-Terrestrial Networks 6G Vision

The forthcoming 6G-enabled Intelligent Transportation System (ITS) is set to redefine conventional transportation networks with advanced intelligent services and applications. These technologies, including edge computing, Machine Learning (ML), and network softwarization, pose stringent requirements for latency, energy efficiency, and user data security. Distributed Learning (DL), such as Federated Learning (FL), is essential to meet these demands by distributing the learning process at the network edge. However, traditional FL approaches often require substantial resources for satisfactory learning performance. In contrast, Transfer Learning (TL) and Split Learning (SL) have shown effectiveness in enhancing learning efficiency in resource-constrained wireless scenarios like ITS. Non-terrestrial Networks (NTNs) have recently acquired a central place in the 6G vision, especially for boosting the coverage, capacity, and resilience of traditional terrestrial facilities. Air-based NTN layers, such as High Altitude Platforms (HAPs), can have added advantages in terms of reduced transmission distances and flexible deployments and thus can be exploited to enable intelligent solutions for latency-critical vehicular scenarios. With this motivation, in this work, we introduce the concept of Federated Split Transfer Learning (FSTL) in joint air-ground networks for resource-constrained vehicular scenarios. Simulations carried out in vehicular scenarios validate the efficacy of FSTL on HAPs in NTN, demonstrating significant improvements in addressing the demands of ITS applications.Comment: 6 pages, 5 figure

Density-Aware Smart Grid Node Allocation in Heterogeneous Radio Access Technology Environments

Smart grid (SG) is an intelligent enhancement of the conventional energy grid allowing a smarter management. In order to be implemented, SG needs to rely on a communication network connecting different node types, implementing the SG services, with different communication and energy requirements. Heterogeneous network (Het-Net) solutions are very attractive, gaining from the allocation of different radio access technologies (RATs) to the different SG node types; however, due to the heterogeneity of the system, an efficient radio resource optimization and energy management are a complex task. Through the exploitation of the most significant key performance indicators (KPIs) of the SG node types and the key features of the RATs, a joint communication and energy cost function are here defined. Through this approach it is possible to optimally assign the nodes to the RATs while respecting their requirements. In particular, we show the effect of different nodes’ density scenarios on the proposed allocation algorithm